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ABSTRACT: Urban school districts face a dilemma in providing professional 
development support for teachers in transition to the Next Generation Science Standards 
(NGSS). Districts need to maximize the quality and amount of professional development 
within practical funding constraints. In this paper, we discuss preliminary results from a 
researcher-practitioner partnership between Northwestern University and the Chicago 
Public Schools. We explore a model for quarterly NGSS professional development for 
urban middle school science teachers that spans three years. The workshop inquiry 
experiences are aligned to areas of research excellence at Northwestern. Teachers in 
twenty-three of the schools responded to a survey on the impact of formal and informal 
learning experiences on changes in teaching practice. We also analyzed the growth in 
student-rated inquiry-based science teaching practices. The results indicate that the 
professional development program had a significant direct impact on teaching practices 
as well as indirect impact on the teaching practices of other teachers at the school. 
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Urban school districts face a dilemma in providing professional development support for 
teachers in transition to the Next Generation Science Standards (NGSS). On the one hand, 
supporting shifts in teaching practice requires significant investment in professional development 
that is (1) long in duration, both in terms of total contact hours and in time span, and (2) involves 
the collective participation of school staff (Desimone & Garet, 2015; Garet, Porter, Desimone, 
Binnan, & Yoon, 2001). Prior research has also shown that the direct benefits of professional 
development can also provide indirect benefits when teachers share what they have learned in 
professional development with other teachers at their school (Penuel, Sun, Frank, & Gallagher, 
2012). On the other hand, frequent teacher turnover in urban settings creates barriers to schools’ 
ability to reap the benefits of their professional development investment. Urban schools need to 
continually invest in new teachers. For example, less than half of Chicago teachers stay at the 
same school for more than four years (Allensworth, Ponisciak, & Mazzeo, 2009). Turnover is 
particularly acute for teachers in high poverty, low performing schools. This instability of the 
teaching corps tends to decrease curriculum coherence and the quality of instruction (Bryk, 
Sebring, Allensworth, Luppescu, & Easton, 2010). In addition, the small size of many schools in 
urban settings often means that there is only one teacher at the school in a given subject area, 
thus reducing the benefits that come from collective participation. In this paper, we discuss 
preliminary results from a researcher-practice partnership (RPP) between Northwestern 
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University and a network of Chicago Public Schools (CPS) elementary schools in which we are 
exploring a model for Next Generation Science Standards (NGSS) professional development that 
is uniquely adapted to an urban context. 

Conceptual Framework 

We need more information about specific aspects of [professional development] 
that are important in different contexts, in order to form a better understanding of 
why some [professional development] works and some doesn’t. (Desimone & Garet, 

2015, p. 260) 

This work is anchored in the literature on teacher professional development. In a recent 
review of professional development research, Desimone and Garet (2015) present a conceptual 
framework highlighting key features for designing professional development to positively impact 
teacher knowledge and changes in inquiry-based teaching. In addition to the two features listed 
above, high quality professional development also (3) is content focused, (4) involves active 
learning, and (5) is coherent with the school goals, teacher knowledge, the needs of students, and 
local policies. The content of professional development in science includes the disciplinary core 
ideas, science practices, and cross-cutting concepts that comprise the NGSS (NGSS Lead States, 
2013). The active learning dimension incorporates four components: planning instruction, giving 
professional presentations, peer observation, and collaborative discussion. The latter two 
activities are often associated with professional community. The Desimone and Garet conceptual 
framework for professional development aggregates the prevalence of these four components of 
active learning into a single active learning scale. This active learning scale aggregates both 
individual and group activities. In addition, this active learning scale does not make a distinction 
between teachers collaborating with colleagues in their school or teachers collaborating with 
peers from other schools participating in the professional development. 

Parise and Spillane (2010) extended the findings of Garet et al. (2001) by investigating 
how changes in teaching practice are differentially influenced by the indicators of professional 
community that are related to the Desimone and Garet (2015)’s active learning scale. Parise and 
Spillane (2010) labeled these indicators of professional community as on-the-job learning 
opportunities (i.e., peer observation, collaborative discussion, and advice seeking). They 
compared these indicators of professional community to fonnal learning opportunities, including 
professional development workshops, coursework, and network participation. Their study 
investigated these indicators for elementary math and English teaching in an urban setting, but 
not for science teachers. Of the various types of fonnal learning opportunities, content-specific 
professional development workshops significantly predicted self-reported changes in teaching 
practices, which is consistent with Garet et al. (2001). In contrast to Garet et al. (2001), Parise 
and Spillane (2010) found that a subset of the indicators of professional community directly 
predicted changes in teaching practices, specifically collaborative discussion and advice seeking. 
Likewise, McGee (2016) replicated the findings from Parise and Spillane, but in the discipline of 
science. He, too, found that collaborative discussion and advice seeking predicted self-reported 
changes in high school science teaching practice. 

In addition to serving as a framework for the design of professional development, 
Desimone (2009) also presents the framework as a means to inform research on professional 
development. By relating the features of specific professional development programs to the 
framework, it becomes possible to aggregate the results of research across programs to draw 
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conclusions on the effects of the various components on teacher knowledge and practice. In 
order to frame the design of our professional development program, we conducted an analysis of 
a small sample of impact studies on NGSS professional development programs. We used three 
sources, which resulted in eight studies. There is one study on science professional development 
cited in Desimone and Garet (2015); (see Penuel, Gallagher, & Moorthy, 2011). We became 
familiar with two programs presented at the 2016 and 2017 National Association of Researchers 
of Science Teaching annual meetings (see Reiser, Michaels, Dyer, Edwards, & McGill, 2016; 
Tuttle et ah, 2016). Lastly, we used the search term “impact of NGSS professional development” 
in Google Scholar and limited the search to 2016-2017. An examination of the first two result 
pages yielded five more studies (see Capraro et ah, 2016; Lotter et al., 2016; Minor, Desimone, 
Lee, & Hochberg, 2016; Taylor, Roth, Wilson, Stuhlsatz, & Tipton, 2017; Yoon et al., 2016). 
We characterized these studies according the Desimone and Garet framework (see Table 1). 

Table 1: Characterization of a sample of science professional development programs by the 
dimensions of the Desimone and Garet framework 


Study 

Duration 

Content Focus 

Coherence 

Active 

Learning 

Collective 

Participation 

Capraro 

60 hrs/year for 

3 years 

Measurement, Math 
problem solving, 
motion and energy. 

Focused on 
district needs 

Design PBL 
modules 

75 teachers 
from 3 high 
schools 

Lotter 

82 hrs/year 

Energy, genetics, 
astronomy, geology 

Locally 

developed units 

Engage in 
inquiry lesson, 
practice teach 
inquiry lessons, 
group reflection 

Statewide 
recruitment 
Teachers signed 
up individually 

Minor 

34 hrs/year for 

2 years 

Content related to 
one unit per grade 
level 

Each unit 
aligned to 
district scope 
and sequence 

Engage in 
inquiry lessons, 
collaborative 
discussion 

Not described 

Penuel 

112 hrs 

Big ideas in Earth 
Science 

Aligned to 
district needs 

Plan for 
instruction, 
collaborative 
discussion 

Grade level 
teams from one 
district 

Reiser 

112 hrs 

3-dimensional 
learning 

Alignment to 
NGSS 

Engage in 
inquiry lessons, 
Examine video 
cases of 
teaching 
practice 

Teachers across 
a state were 
selected by 
local districts to 
participate 
locally 

Taylor 

88.5 hrs 

4 th grade: Earth 

processes, 

ecosystems 

5 th grade: Climate, 
water cycle 

Aligned to state 
standards 

Examine videos 
of others and 
own practice, 
design a unit 

School teams 
across midrange 
districts in CO 

Tuttle 

80 hrs/year 

One discipline per 
grade band 

Aligned to 

NGSS 

Engage in 
inquiry lessons, 
plan instruction 

Recruited 
individually 
from an urban 
district 
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40 hrs/year for 
two years 


Focused on 

Engage in units, 

10 teachers 
from 7 schools 

Yoon 

5 biology units 

Complex 

systems 

plan for 
instruction 

in a 

metropolitan 

area 


A core feature of the framework is engaging teachers in professional development of 
sustained duration. While research does not point to specific guidelines for the length of 
professional development, Desimone and Garet (2015) have found that a minimum of 20 hours 
of professional development is needed to positively impact complex changes in teaching 
practice, such as inquiry-based science instruction. Amongst the sample programs reviewed, all 
of them provide more than 20 hours of professional development per year, with the minimum 
providing 34 hours in one year (Minor et ah, 2016) and the maximum providing 112 hours in one 
year (Penuel et ah, 2011; Reiser et al., 2016). Most of the sample programs provided more than 
50 hours of professional development per year for multiple years. 

Professional development should focus on specific content and be coherent with the needs 
of teachers and school districts. All of the sample programs focused on science content. Three of 
the sample programs indicated that they were working with specific school districts. Of those 
three, two indicated that the choice of science content was made in conjunction with the school 
district (Capraro et al., 2016; Penuel et al., 2011). Only one of those three programs indicated 
that the participating schools served primarily low-income students (Capraro et al., 2016). Of the 
programs that recruited teachers by region or statewide, only one provided a rationale for the 
selection of specific content, which was based on the goal of promoting systems thinking in 
biology (Yoon et al., 2016). Since the program recruited regionally, the teachers who joined 
presumably felt that the systems thinking content was coherent with their own goals. The other 
programs aligned their content to state or national standards. 

The collective participation of school teams affords opportunities for teachers to more 
easily benefit from collaboration around the implementation of what was learned in the 
professional development. Only three projects specifically mentioned that teachers were 
recruited as school teams to participate in the workshops (Capraro et al., 2016; Penuel et al., 
2011; Taylor et al., 2017). In the cases of Capraro et al. (2016) and Penuel et al. (2011), the 
professional development was conducted in conjunction with specific school districts. While not 
explicitly mentioned, the size of the teacher population and participation by school teams implies 
that teachers in these two projects were expected to participate in the professional development. 
All of the other programs seemed to recruit teachers on a voluntary basis, which means they are 
more likely to be actively engaged and reap the benefits of the program since they chose to 
participate. 

The modes of engaging teachers in active learning varied. Some programs provided 
teachers with instructional materials. These programs prepared teachers by having them engage 
in the activities and then plan for how they would implement those activities in their own 
classroom (Lotter et al., 2016; Minor et al., 2016; Penuel & Gallagher, 2009; Yoon et al., 2016). 
Other programs engaged teachers in lesson/unit development (Capraro et al., 2016; Penuel & 
Gallagher, 2009; Tuttle et al., 2016). A handful of programs engaged teachers in reflection on 
videos of science teaching practices (Reiser et al., 2016; Taylor et al., 2017). In general, 
professional development programs, such as these, influence teachers’ knowledge and skills, 
which in turn influence teaching practices (Desimone, 2009). 
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Professional Development Needs of Urban Districts 

Desimone and Garet (2015) caution the field that the professional development needs of 
urban districts provide unique challenges for their framework. The high turnover of teachers and 
principals in urban settings creates challenges for both implementing and conducting research on 
professional development programs in urban context. Of the sample of recent studies we 
reviewed on impact of NGSS professional development programs, only one specifically targeted 
a low-income context (Capraro et ah, 2016). The program had unusually high teacher retention, 
with most of the sample of 75 teachers across three high schools completing the three-year 
program. However, the researchers had to drop one-fourth of the student sample since they were 
not in the same high school for the three-year study. In their section on the challenges of urban 
professional, Desimone and Garet (2015) did not cite any studies on professional development in 
urban settings. We need to find ways to design, implement, and conduct research on professional 
development programs that do not ignore our most vulnerable students. 

NU Leadership Academy Professional Development Model 

In our CPS context, high teacher turnover, the small size of neighborhood schools, and the 
fiscal crisis in IL create barriers to realizing the duration, coherence, and collective participation 
that is consistent with other science professional development programs, as represented by the 
sample programs reviewed in this paper. All of the programs reviewed here included a summer 
professional development experience. The cost to CPS for summer stipends alone for the 
targeted network of schools would be over $100,000. On average, about $20,000 of that 
investment would be lost at the end of each year when one-fifth of the teachers leave the schools. 
In addition, anecdotal conversations with our teachers indicate that many of them use the 
summer to decompress from the intensity of their work environment and refuse to participate in 
summer professional development. The timing and duration of the workshop experiences need to 
provide a sustainable model for urban districts. Our professional development includes quarterly 
one-day, seven-hour workshops during school days. Each participating school is responsible for 
funding the substitute teacher. The combination of reducing the number of hours and shifting of 
costs from stipends to subs significantly reduces the costs for the district in comparison to a 
summer program. If CPS feels that the model is successful, it could easily be adapted to fit the 
context of the ongoing quarterly in-service professional development days, which would reduce 
the costs even further. At 28 contact hours per year, our program is above the 20-hour 
benchmark recommended by Desimone and Garet (2015), but 20% below the shortest duration 
program reviewed above. 

Frequent teacher turnover also creates barriers to maintain coherence across years of a 
professional development program. Several of the programs reviewed here are multi-year 
programs that are cumulative. Those programs seemed to benefit from settings with a stable 
cadre of teachers or perhaps, the voluntary nature of the professional development attracts 
teachers who are more likely to stay at their schools. In a context like Chicago, a coherent 
program is one that can provide an accumulation of knowledge and skills for those that 
participate across multiple years, but each session is also stand-alone such that teachers can join 
the program at any point. In addition, the sustained fiscal crisis in Chicago has prevented schools 
from replenishing science curricula for almost a decade. Therefore, a coherent program needs to 
support implementation of freely available resources across the widest possible topic areas 
within the duration constraints. This model of coherence seems to contrast with those programs 
reviewed here, in which the focus was fewer topic areas in greater depth. It is likely that the 
middle- to upper-income districts that are typically the target of science professional 
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development research already have current science curricula in place and can benefit from a deep 
focus in a subset of the school year curriculum. 

The NU Leadership Academy model is a three-year professional development program 
targeted at the middle school teachers in a network of thirty-three K-8 elementary schools in 
CPS. The student population in this network is 95% African American and 90% low-income as 
indicated by participation in free and reduced price lunch programs. For most of the schools in 
the network, there is only one middle school teacher who teaches 6th, 7th, and 8th grade science. 
We were able to secure funding from the Fry Foundation and the Toyota Foundation to cover the 
expenses of planning, implementing, and evaluating the quarterly professional development 
program. Through this cohort model, teachers had the opportunity to collaborate with teachers at 
other schools serving similar populations of students. While not from the same school, this 
model has the potential to provide some of the benefits of collective participation. The schools 
provided the funds for substitute teachers. This shared cost model also placed an incentive for 
principals to hold teachers accountable for implementing what was learned. We also have 
anecdotal evidence that some principals required teachers to attend the professional development 
workshops. Therefore, we ended up with a mixture of volunteers and “voluntold” participants. 

We discussed three criteria for the selection and sequence of topics to maximize coherence 
within the duration constraints. First, the workshop topics align to the district's science scope and 
sequence, such that the middle school teachers participate in workshops in preparation for the 
NGSS performance expectations to be addressed in that quarter. Second, the workshops focus on 
one grade level each year, with 7th grade life science in 2015-16, 6th grade Earth science 
planned for 2017-18, and 8th physical science planned for 2018-19. Since there is primarily one 
teacher for the three grade levels, teachers who remain in their school for the three years will 
benefit from the accumulated experience of sixteen NGSS-related workshops over three years. 
For schools where there is teacher turnover, new teachers can join the series and still 
productively participate since each year and each workshop stands alone. In addition, we 
identified those teachers who consistently participated in workshops and who provided evidence 
that they were implementing was learned. These active participants were invited to become co¬ 
designers and co-facilitators of subsequent workshops as a way to develop capacity within the 
network to sustain the workshops in future years. Third, the targeted workshop topics also align 
to the areas of research excellence at Northwestern University. Science and engineering graduate 
student researchers from those areas of excellence worked with us to create research experiences 
for workshop participants. These research experiences provide a model for specific units that 
teachers can implement. In addition, these inquiry-based activities challenge the middle school 
teachers' understanding of core disciplinary ideas and scientific practices in NGSS. 

For example, in the first quarter of the district’s 7th grade life science scope and sequence, 
the focal disciplinary core ideas are related to From Molecules to Organisms: Structures and 
Processes. Within that broad DCI, there was one performance expectation that aligned to a 
neuroscience program at the university. MS-LS1-8 focuses on gathering and synthesizing 
information about how sensory receptors send messages to the brain for immediate behavior. 
Two neuroscience graduate students worked with us to design the workshop experiences. In the 
investigation, teachers used a web-based reaction time system to design experiments to explore 
factors that affect reaction time (http://cognitivefun.net/test/l). For example, teachers examined 
gender, dominant vs non-dominant hand, and with vs. without glasses. These experiments 
provided the backdrop for gathering and synthesizing information about a particular neurological 
disease, Guillain-Barre Syndrome. Teachers provided an explanation for how the disease would 



JUL.TR, 2017, 79 


impact messages from the sensory receptors and affect reaction time. After completing the 
investigation, the teachers were provided time to develop plans for implementing the 
investigation with their students. The workshops in subsequent quarters followed a similar 
pattern for topics in genetics, ecology, and evolution. While the specific science and engineering 
practices and cross-cutting concepts varied depending on the target performance expectations, 
we maintained a focus on science explanation as a core practice as well as implemented the 
network’s overall teaching strategies related to reading-to-leam. 

In this study, we sought to investigate what level of impact our professional development 
model had on teachers’ practices as self-reported by the teachers as well as reported by the 
students. Desimone (2009) recommends that professional development researchers also measure 
changes in teacher content knowledge and pedagogical content knowledge as mediators of 
changes in teaching practice. However, given the variability of participation in the workshops 
across the thirty-three schools and the limited resources of the professional development 
program, it was not viable to measure changes in teacher knowledge. Desimone (2009) also 
recommends that professional development programs measure the impact of changes in teaching 
practice on student learning. At the time of this study, the state of IL was in transition to a new 
NGSS-aligned assessment. Therefore, there was no state student science assessment 
administered during the timeframe of this study. However, research similar to Parise and 
Spillane (2010) has shown that self-reported changes in teaching practice are predictive of 
improvements in student learning (Supovitz, Sirinides, & May, 2010). Therefore, this study 
focuses solely on the impact of participation in the professional development on teaching 
practices. The time frame of this study extends from the 2012-13 school year, which is the year 
before the schools were formed into a network, through the 2013-14 school year, when we pilot 
tested a version of the model, to the 2015-16 school year, which was the first year of the current 
model focusing on 7 th grade life science topics. (The 2014-15 school year was a gap in the 
implementation of the model.) This study will address the following research questions: 

• Does the level of participation NU Leadership Academy predict changes in 
teaching practice? 

• Does active participation in the NU Leadership Academy provide indirect benefit to 
other teachers at the schools where active participants teach? 

Methods 

In this research, we investigate whether the level of participation in the workshops 
influenced teachers' changes in teaching practices, both self-reported and as reported by students. 
Our approach parallels the research design used by Parise and Spillane (2010). They extended 
the findings of Garet et al. (2001) by investigating how changes in elementary math and English 
teaching practice are differentially influenced by formal and informal professional learning 
opportunities. Parise and Spillane examined the impact of coursework, professional development 
and participation in outside networks. Of these various types of fonnal learning opportunities, 
only content-specific professional development significantly predicted changes in teaching 
practices, which is consistent with Garet et al. (2001). Parise and Spillane also examined 
informal professional learning opportunities (i.e., peer observation, collaborative discussion, and 
advice seeking) and found that the extent to which teachers engaged in collaborative discussion 
with colleagues and sought advice from colleagues predicted changes in teaching practice. In 
previous research in CPS, we replicated the findings of Parise and Spillane in the absence of any 
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external professional development program for high school science departments in a network of 
CPS high schools (McGee, 2016). 

Measures 

There are two sources of outcome data about teaching practices. The first source is a 
variation of School Staff Questionnaire used in Parise and Spillane (2010). The survey was 
administered at the end 2015-16 school year and only addresses the impact of the first year of the 
program on teaching practices. In addition, we gathered district data about the level of inquiry- 
based science teaching as reported by students on the district’s 5 Essentials questionnaire (Bryk 
et al., 2010). These data cover the timespan of the whole study period from 2012-13 to 2015-16. 

School Staff Questionnaire. This questionnaire was adapted from Parise and Spillane 
(2010). The questionnaire contains the dependent variable related to self-reported changes in 
teaching practice. The independent variables distinguish between the level of participation in 
formal and informal professional learning opportunities. The formal professional learning 
opportunities items include participation in workshops, graduate coursework and participation in 
networks. By asking teachers in general about the amount of workshop participation it allows us 
to capture not only participation in the NU Leadership Academy workshops, but also other 
professional development opportunities that may be available. The level of network participation 
provides an indicator of the extent to which teachers may view their participation in the NU 
Leadership Academy as a form a collective participation. 

While the NU Leadership Academy did not provide specific supports for informal 
professional learning at the schools, our prior research in Chicago indicates that there is a 
prevalence of informal learning occurring in CPS schools without specific external supports 
(McGee, 2016). In addition, by surveying other teachers at the schools, primarily K-5 teachers, 
we can address our second research question on the extent to which there are indirect benefits to 
other teachers at the school. The infonnal professional learning constructs are collaborative 
discussion, peer observation, and advice seeking. The questionnaire also includes control 
variables related to teacher demographics. The survey contains twenty-six questions, which took 
about 15 minutes to complete. 

Changes in science teaching practices scale (dependent variable). The dependent measure 
is a scale comprised of eight questions about the extent to which teachers changed their science 
teaching practice in the past year on a seven-point scale for the following aspects of science 
teaching: (1) student assessment, (2) student grouping, (3) materials used, (4) topics covered, (5) 
teaching methods used, (6) kinds of work students do, (7) kinds of questions asked, and (8) 
understanding the needs of individual students in their class. The items were averaged to create 
the changes in science teaching practices variable, which served as a dependent variable for this 
study. The alpha reliability of the changes in science teaching practices variable was 0.93. 

Formal professional learning opportunities (independent variables). There were three 
questions about fonnal professional learning opportunities. First is frequency of participation in 
science professional development workshops in the last year. The answer options for workshops 
are on the following scale: None (1), 1-2 sessions (2), 3-4 sessions (3), 5-7 sessions (4), and 
8+sessions (5). Second is the number of science or science education courses taken in the last 
year. The answer options for science coursework are on the following scale: None (1), 1 class 
(2), 2 classes (3), 3 classes (4), and 4+ classes (5). Third is the frequency of outside network 
participation in the last year. The answer options for network participation are on the following 
scale: Never (1), 1-2 times (2), 3-4 times (3), and 5+ times (4). Each question will be used 
separately as an independent variable. 
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Informal professional learning opportunities (independent variables). There were three 
constructs related to the prevalence of informal professional learning opportunities: collaborative 
discussion, peer observation, and advice seeking. To capture the range of informal professional 
learning opportunities, each construct contains multiple questions that form into a scale. All of 
the answer options are on the following scale: Never (1), A few times per year (2), Monthly (3), 
Weekly (4), and Daily (5). 

The collaborative discussion dimension contains seven questions that measure the self- 
reported frequency with which teachers engage in conversation with colleagues regarding 
teaching and learning: (a) what helps students learn the best, (b) development of new curriculum, 
(c) the goals of this school, (d) managing classroom behavior, (e) science instruction, (f) content 
or performance standards in science and (g) collaborative review of student work. The alpha 
reliability of the collaborative discussion scale as a whole was 0.87. 

There were three questions about the frequency with which teachers engage in peer review 
and feedback. The alpha reliability of the peer observation and feedback scale was 0.80. 

There was one question related to science advice seeking. Teachers were asked, “To whom 
do you turn for advice or infonnation about science instruction?” Respondents could list up to 
ten different individuals who served as sources of advice. As an indicator of strength of the 
relationship with each advice giver, teachers were also asked to indicate how often they turned to 
each source for advice, ranging from yearly to daily. The science advice seeking measure is 
created by totaling the frequency with which advice was sought from all sources listed. The 
science advice giving construct is created by totaling the frequency with which others sought 
advice from a teacher. 

Teacher characteristics (control variables). There were five questions about individual 
teacher characteristics, which were included as control variables in the analyses, including age, 
number of years as a teacher, number of years teaching at the current school, gender, and race. 

Inquiry-based Science Instruction Scale. Each year, the 5 Essentials survey is 
administered to every teacher in CPS and to every middle school and high school student in CPS. 
The overall survey addresses a variety of school climate topics. Within the overall 5 Essentials 
survey for students, there is an Inquiry-based Science Instruction scale. Students report the 
frequency to which they: use laboratory equipment or specimens, write lab reports, generate your 
own hypotheses, use evidence/data to support an argument or hypothesis, and find information 
from graphs and tables. The answer options are on the following scale: Never, Once or twice a 
semester, Once or twice a month, Once or twice a week, and Almost every day. Researchers at 
the University of Chicago use hierarchical linear scaling to develop a scale score for each school 
along the following range: Very Weak (1), Weak (2), Neutral (3), Strong (4), and Very Strong 
(5). The survey scale has been validated to predict growth in ACT performance (Allensworth, 
Correa, & Ponisciak, 2008). 

Population 

During the months of May and early June 2016, we used two approaches for administering 
the School Staff Questionnaire. In the first approach, all of the middle school teachers who 
attended the fourth quarter workshop in spring 2016 completed the survey online at the end of 
the workshop experience. In addition, the teachers were provided with a package of paper 
surveys. They were asked to distribute the surveys to the K-5 teachers at their school, collect the 
completed surveys, and mail the package back to the network office. For schools that did not 
have a teacher attend the fourth quarter workshop, packages of paper surveys were sent to the 
middle school teachers. They were asked to complete a survey, distribute surveys to the K-5 
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teachers at their schools, collect the completed surveys, and mail the packages back to the 
network office. The teachers who were provided with the surveys at their schools also had the 
option of completing the survey online. 

The teachers in this study come from 23 of 33 schools in the network. A total of 93 
teachers responded to the survey. Thirty (30) middle school science teachers at the 23 schools 
responded to the survey, reflecting that some schools have more than one middle school science 
teacher. Of the middle school teachers, 12 were active participants in the workshops during the 
2015-16 and 18 did not actively participate in the program. These 12 active participants engaged 
in more than 20 hours of professional development, which is above the Desimone and Garet 
(2015) threshold. Sixty-three (63) K-5 teachers out of roughly 560 K-5 teachers at the 23 schools 
responded to the survey (12% response rate). These K-5 responses are distributed across both 
schools with active participants and those without active participants. The significantly higher 
response rate for middle school teachers versus elementary teachers reflects the fact that the 
intervention was targeted at middle school teachers. However, even though the K-5 teacher 
response rate was low overall, there are a sufficient number of respondents to allow us to draw 
conclusions about the indirect benefits of the middle school professional development on K-5 
teachers. 

Three-fourths of the teachers identified as African American (74%) and about one-fifth 
identified as Caucasian (22%). The remaining teachers identified as Hispanic (3%), and Other 
(4%). Ninety-one percent of the teachers were female (91%). The distribution of overall teaching 
experience is skewed towards teachers with more than 10 years of experience (63%). The 
median years of teaching experience was 15 years (see Table 2). However, the distribution of 
experience at the current school was skewed toward the lower end, with half of the teachers at 
their school for five years or less (51%) and the median years at the current school was 5 years. 
This discrepancy between overall teaching experience and experience at the current school 
suggests that the population reflects the high turnover rate typical of urban districts (Allensworth 
et al., 2009). 

Table 2: Years of Teaching Experience 



At 

School 

Total 

1-5 yrs 

46 

12 

6-10 yrs 

21 

21 

11-15 yrs 

10 

21 

16-20 yrs 

8 

22 

>20 

3 

15 


Results 

Table 3 compares the descriptive statistics between the elementary teachers and the middle 
school teachers for the primary dependent and independent variables included in this study. For 
each variable, we conducted a one-way ANOVA to test the statistical significance of the mean 
differences by group. The dependent variable—changes in science teaching—indicates that, on 
average, elementary teachers were slightly below the midpoint of 4 on the scale. Middle school 
teachers who were active participants were statistically more likely to have made changes to their 
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teaching practice at 6 on a 7-point scale (F(2,85)=12.5; pO.OOOl). Non-active middle school 
teachers averaged at midpoint. In terms of formal professional learning opportunities, elementary 
teachers on average participated in 1-2 science workshops. Active middle school teachers on 
average attended 5-7 science workshops, whereas non-active middle school teachers on average 
attended 3-4 science workshops (F(2,88)=32.9; p<0.0001). In addition, active middle school 
teachers were more likely to report participation in outside network activities between 3 to 4 
times per year on average (F(2,90)=45.3; p<0.0001). These results provide evidence that active 
teachers may be viewing the cohort-based workshops as an opportunity to collaborate with 
teachers across schools and to possibly accrue some of the benefits of collective participation. 

Both active and not active middle school teachers were more likely to engage in 
collaborative conversations with their colleagues (about once per month), than elementary 
teachers (between a few times per year and once per month) (F(2,90)=4.9; p=0.01). There was no 
statistically significant difference between elementary and middle school teachers on the 
frequency of advice seeking or peer observation and feedback. However, there was a statistically 
significant difference in the extent to which active middle school teachers were sought out for 
advice by colleagues (F(2,90)=3.3; p<0.05). Active middle school teachers were more than twice 
as likely to be sought out for advice. 


Table 3: Descriptive statistics for the dependent and independent variables 


Variable 

Elementary 

Middle 

School 

Middle 
School Active 

p value 

N 

61 

18 

12 


Dependent Variable 

Changes in science teaching 

3.4 

4.0 

6.0 

0.000 

Formal Learning Opportunities 

Science professional development 

1.7 

2.6 

4.0 

0.000 

Science Coursework 

1.2 

1.2 

1.4 

ns 

Network participation 

1.4 

1.9 

3.5 

0.000 

Informal Learning Opportunities 

Conversations with colleagues 

2.6 

3.0 

3.2 

0.01 

Science Advice Seeking 

2.7 

2.5 

3.7 

ns 

Give Science Advice 

1.0 

0.4 

2.3 

0.04 

Peer Observation and Feedback 

1.1 

1.4 

1.3 

ns 


Regression Results of School Staff Questionnaire 

As indicated above, this research parallels the work of Parise and Spillane (2010). 
Therefore, we followed their analytic strategy for modeling the impact of formal and infonnal 
learning opportunities on changes in teaching practices. We used multiple stepwise regression to 
independently examine the influence of formal learning and infonnal learning opportunities on 
changes in teaching practice and then examined the combined influence of both fonnal and 
informal learning opportunities. All of the variables for a given model were entered into the 
regression. The variable that provides the most infonnation is added to the equation and the 
regression is run again to add the variable that provides the next most amount of infonnation 
until the equation includes only variables that are statistically significant. Table 4 shows the 
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variables that were included for each of the three models. Variables that were not entered into the 
model are indicated as NS for not significant. 

The regression model for Formal Learning Opportunities was statistically significant 
(F(l,82)=64.13; p<0.001; R 2 = 43%). The only statistically significant variable was the extent to 
which teachers participated in science professional development. The regression model for 
Infonnal Learning Opportunities was statistically significant (F(2,85)=16.33; p<0.001; R 2 = 
26%). There were two statistically significant variables. The extent to which teachers engaged in 
conversations about science influenced teachers’ science teaching practice. Consistent with prior 
research showing the indirect benefits of professional development, advice seeking at schools 
with an active middle teacher was associated with changes in science teaching practice. The 
combined model was statistically significant (F(3,79)=32.71; p<0.001; R 2 = 54%). Attendance at 
professional development and school-based conversations were statistically significant. Advice 
seeking at schools with an active middle school teacher was not statistically significant in the 
combined model. However, when accounting for variability in science professional development 
attendance, the conversations at schools with active middle school teachers had additional 
influence on changes in science teaching. 

Table 4: Results of Multiple Regression of Variation of Learning Opportunities on 
Changes in Science Teaching Practice 


Independent Variable 

Changes in Science Teaching Practice 

Formal Learning Informal Learning 

Opportunities Only Opportunities Only Full Model 

Formal Professional Opportunities 

Science Professional Development 

0.52** 


0.56** 

PD X Grade Level 

NS 


NS 

Science Courses (School Year) 

NS 


NS 

Outside Network Participation 

NS 


NS 

Informal Professional Opportunities 

Conversations about science 


0.48** 

0.27** 

Conversations @ Active School 


NS 

0.16* 

Advice Seeking 


NS 

NS 

Advice Seeking @ Active School 


0.19* 

NS 

Peer observation and feedback 


NS 

NS 

Teacher Controls 

Years Teaching at Current School 

NS 

NS 

NS 

Gender, Race 

NS 

NS 

NS 

R-square 

43% 

26% 

54% 

Observations 

84 

88 

83 


Significance Levels: 0.05 = *; 0.01 = **; NS = not significant 


Growth of Science Teaching Practices by Level of Workshop Participation 

The regression results of the School Staff Questionnaire show that level of participation in 
in one year of professional development influenced the amount of self-reported changes in 
teaching practices over the course of one year. To corroborate those findings, we conducted an 
analysis of the influence of workshop participation on the year over year changes in inquiry- 
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based science teaching as rated by the students. This analysis included the school level aggregate 
ratings for all thirty-three elementary schools in the network over four years. The 2012-13 school 
year serves as a baseline since it is the year before the schools in CPS were reorganized into new 
networks. We provided a pilot version of the professional development program in 2013-14 and 
the first year of the full model in 2015-16. The 2014-15 school year was a gap year for the 
professional development intervention. Schools were categorized as active in a given year if the 
schools sent teachers to more than half of the workshops that year, even if the school sent 
different teachers to different workshops. By shifting the analytic approach from the individual 
teachers to the school level effects, it allows us to analyze the impact of the program despite 
teacher turnover. 

Given that there were four measurement occasions for each of the schools, we used a 
hierarchical linear model to analyze changes in student rated inquiry-based science teaching. The 
measurement occasions served as the level 1 variable. The level of active participation at the 
school level (none, 1 year, 2 years) served as a level 2 variable. The baseline growth model was 
not statistically significant (t(32)=1.7; ns). There was an interaction effect between the growth 
curve and the number of years that a school actively participated in the NU Leadership Academy 
(t(32)=3.9; p<0.001). The addition of level of participation into the model reduced the variance 
around the slope of the growth in inquiry-based science teaching by 10%. Each year of active 
participation increased the growth rate by 0.15 points on the index, such that schools that were 
active for both years grew at a rate of 0.30 points per year over the course of the study period. 
There was no statistically significant difference in the inquiry-based science teaching scale in the 
baseline year. Figure 1 provides a graphical display of these results. The graph was generated 
based on the average intercept and slope for each condition. 

Figure 1 


Growth in Inquiry-Based Science Instruction by 
Level of Participation in NU Leadership Academy 
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Discussion and Conclusion 

In this research, we developed a professional development model based on Desimone and 
Garet (2015) that addressed the unique constraints of urban contexts. The levels of high teacher 
turnover, the small size of neighborhood schools, and the fiscal crisis in IL create barriers to fully 
realizing the duration, coherence, and collective participation dimension in urban contexts. In 
order to minimize the costs of the program to the district, we provided quarterly school year 
workshops such that the number of hours per year was above the threshold of 20 hours, but 20% 
below comparable models. To foster coherence, we aligned the quarterly workshops to the 
specific performance expectations that teachers were expected to be teaching around the time of 
the workshop. The workshops were designed to provide teachers with experience on specific 
lessons to be implemented. The workshop experiences were designed to focus on explanation as 
a core practice, thus enabling the accumulation of knowledge across workshops. At the same 
time, each workshop was designed to be stand-alone so that new teachers could join the program 
at any point. 

In this research, we sought to investigate whether the level of participation the NU 
Leadership Academy influences changes in teaching practice. The results of analysis of the 
School Staff questionnaire showed that teachers who were active participants in the academy 
participated in more professional development overall than other middle school teachers and K-5 
teachers in the same network. The level of participation in professional development was indeed 
a significant predictor of self-reported changes in teaching practice. All teachers in CPS had 
access to other general professional development about NGSS, which also had an impact on their 
changes to teaching practice, but to a lesser degree. We also examined whether participation in 
the academy workshops influenced the extent to which teachers changed their teaching practices 
over time as rated by the students. Over the course of three years, schools that have active 
participants in both years of the academy saw a statistically significantly higher growth rate in 
inquiry-based teaching practices. They went from a neutral rating of inquiry-based teaching 
practices to a strong rating over the course of three years. 

In addition to analyzing the direct benefits of the program, we also investigated the indirect 
effects of the professional development program on other teachers at schools with active 
participants. After controlling for the effects of fonnal professional development workshops, all 
teachers benefited from collaborative conversations. However, collaborative conversations at 
schools that had a middle school teacher who actively participated in the academy had a greater 
influence on changes in teaching practice. In addition, the teachers who actively participated in 
the academy were more likely to be sought out for advice, which is consistent with prior research 
indicating that teachers with specific expertise are more likely to be sought out for advice 
(Spillane, Hallett, & Diamond, 2003). These results provide evidence that there are indirect 
benefits of workshop participation to improve teaching practices for teachers who did not engage 
in the workshop. 

This research points to the promise of engaging teachers in economical professional 
development using a framework of best practices in professional development to design 
professional development that addresses the unique barriers faced by urban school districts. 
While more professional development would likely lead to more significant changes in teaching 
practice, there are practical constraints in urban settings. These results provide evidence that 
school districts can develop a coherent program of science professional development experiences 
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that fit within their existing in-service professional development framework. 

Our future research will continue to track the development of teaching practices as teachers 
continue in the program for additional years. In addition, future research would benefit from 
direct measures of changes in teacher knowledge as a mediator of changes in teaching practice as 
well as the impact of changes in teaching practice on student learning outcomes. 
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